Biased binomial assessment of cross-validated estimation of classification accuracies illustrated in diagnosis predictions
Identifieur interne : 000B32 ( Main/Exploration ); précédent : 000B31; suivant : 000B33Biased binomial assessment of cross-validated estimation of classification accuracies illustrated in diagnosis predictions
Auteurs : Quentin Noirhomme [Belgique] ; Damien Lesenfants [Belgique] ; Francisco Gomez [Colombie] ; Andrea Soddu [Canada] ; Jessica Schrouff [Belgique, États-Unis] ; Gaëtan Garraux [Belgique] ; André Luxen [Belgique] ; Christophe Phillips [Belgique] ; Steven Laureys [Belgique]Source :
- NeuroImage : Clinical [ 2213-1582 ] ; 2014.
English descriptors
- KwdEn :
- MESH :
- diagnosis : Brain Injuries.
- pathology : Brain.
- Adult, Aged, Bias (Epidemiology), Brain-Computer Interfaces, Computer Simulation, Humans, Middle Aged, Models, Statistical, Young Adult.
Abstract
Multivariate classification is used in neuroimaging studies to infer brain activation or in medical applications to infer diagnosis. Their results are often assessed through either a binomial or a permutation test. Here, we simulated classification results of generated random data to assess the influence of the cross-validation scheme on the significance of results. Distributions built from classification of random data with cross-validation did not follow the binomial distribution. The binomial test is therefore not adapted. On the contrary, the permutation test was unaffected by the cross-validation scheme. The influence of the cross-validation was further illustrated on real-data from a brain–computer interface experiment in patients with disorders of consciousness and from an fMRI study on patients with Parkinson disease. Three out of 16 patients with disorders of consciousness had significant accuracy on binomial testing, but only one showed significant accuracy using permutation testing. In the fMRI experiment, the mental imagery of gait could discriminate significantly between idiopathic Parkinson's disease patients and healthy subjects according to the permutation test but not according to the binomial test. Hence, binomial testing could lead to biased estimation of significance and false positive or negative results. In our view, permutation testing is thus recommended for clinical application of classification with cross-validation.
Url:
DOI: 10.1016/j.nicl.2014.04.004
PubMed: 24936420
PubMed Central: 4053638
Affiliations:
Links toward previous steps (curation, corpus...)
- to stream Pmc, to step Corpus: 000A43
- to stream Pmc, to step Curation: 000A43
- to stream Pmc, to step Checkpoint: 000524
- to stream PubMed, to step Corpus: 000768
- to stream PubMed, to step Curation: 000768
- to stream PubMed, to step Checkpoint: 000768
- to stream Ncbi, to step Merge: 001796
- to stream Ncbi, to step Curation: 001796
- to stream Ncbi, to step Checkpoint: 001796
- to stream Main, to step Merge: 000B36
- to stream Main, to step Curation: 000B32
Le document en format XML
<record><TEI><teiHeader><fileDesc><titleStmt><title xml:lang="en">Biased binomial assessment of cross-validated estimation of classification accuracies illustrated in diagnosis predictions</title><author><name sortKey="Noirhomme, Quentin" sort="Noirhomme, Quentin" uniqKey="Noirhomme Q" first="Quentin" last="Noirhomme">Quentin Noirhomme</name><affiliation wicri:level="4"><nlm:aff id="aff1">Cyclotron Research Centre, University of Liège, Liège, Belgium</nlm:aff><country xml:lang="fr">Belgique</country><wicri:regionArea>Cyclotron Research Centre, University of Liège, Liège</wicri:regionArea><orgName type="university">Université de Liège</orgName><placeName><settlement type="city">Liège</settlement><region type="region" nuts="1">Région wallonne</region><region type="province" nuts="1">Province de Liège</region></placeName></affiliation><affiliation wicri:level="1"><nlm:aff id="aff2">Coma Science Group, Neurology Department, University Hospital of Liège, Liège, Belgium</nlm:aff><country xml:lang="fr">Belgique</country><wicri:regionArea>Coma Science Group, Neurology Department, University Hospital of Liège, Liège</wicri:regionArea><wicri:noRegion>Liège</wicri:noRegion></affiliation></author><author><name sortKey="Lesenfants, Damien" sort="Lesenfants, Damien" uniqKey="Lesenfants D" first="Damien" last="Lesenfants">Damien Lesenfants</name><affiliation wicri:level="4"><nlm:aff id="aff1">Cyclotron Research Centre, University of Liège, Liège, Belgium</nlm:aff><country xml:lang="fr">Belgique</country><wicri:regionArea>Cyclotron Research Centre, University of Liège, Liège</wicri:regionArea><orgName type="university">Université de Liège</orgName><placeName><settlement type="city">Liège</settlement><region type="region" nuts="1">Région wallonne</region><region type="province" nuts="1">Province de Liège</region></placeName></affiliation><affiliation wicri:level="1"><nlm:aff id="aff2">Coma Science Group, Neurology Department, University Hospital of Liège, Liège, Belgium</nlm:aff><country xml:lang="fr">Belgique</country><wicri:regionArea>Coma Science Group, Neurology Department, University Hospital of Liège, Liège</wicri:regionArea><wicri:noRegion>Liège</wicri:noRegion></affiliation></author><author><name sortKey="Gomez, Francisco" sort="Gomez, Francisco" uniqKey="Gomez F" first="Francisco" last="Gomez">Francisco Gomez</name><affiliation wicri:level="1"><nlm:aff id="aff3">Complexus Group, Computer Science Department, Universidad Central de Colombia, Bogotá, Colombia</nlm:aff><country xml:lang="fr">Colombie</country><wicri:regionArea>Complexus Group, Computer Science Department, Universidad Central de Colombia, Bogotá</wicri:regionArea><wicri:noRegion>Bogotá</wicri:noRegion></affiliation></author><author><name sortKey="Soddu, Andrea" sort="Soddu, Andrea" uniqKey="Soddu A" first="Andrea" last="Soddu">Andrea Soddu</name><affiliation wicri:level="1"><nlm:aff id="aff4">Department of Physics & Astronomy, Brain and Mind Institute, University of Western Ontario, London, ON, Canada</nlm:aff><country xml:lang="fr">Canada</country><wicri:regionArea>Department of Physics & Astronomy, Brain and Mind Institute, University of Western Ontario, London, ON</wicri:regionArea><wicri:noRegion>ON</wicri:noRegion></affiliation></author><author><name sortKey="Schrouff, Jessica" sort="Schrouff, Jessica" uniqKey="Schrouff J" first="Jessica" last="Schrouff">Jessica Schrouff</name><affiliation wicri:level="4"><nlm:aff id="aff1">Cyclotron Research Centre, University of Liège, Liège, Belgium</nlm:aff><country xml:lang="fr">Belgique</country><wicri:regionArea>Cyclotron Research Centre, University of Liège, Liège</wicri:regionArea><orgName type="university">Université de Liège</orgName><placeName><settlement type="city">Liège</settlement><region type="region" nuts="1">Région wallonne</region><region type="province" nuts="1">Province de Liège</region></placeName></affiliation><affiliation wicri:level="1"><nlm:aff id="aff5">Laboratory of Behavioral and Cognitive Neurology, Stanford University, Palo Alto, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>Laboratory of Behavioral and Cognitive Neurology, Stanford University, Palo Alto</wicri:regionArea><wicri:noRegion>Palo Alto</wicri:noRegion></affiliation></author><author><name sortKey="Garraux, Gaetan" sort="Garraux, Gaetan" uniqKey="Garraux G" first="Gaëtan" last="Garraux">Gaëtan Garraux</name><affiliation wicri:level="4"><nlm:aff id="aff1">Cyclotron Research Centre, University of Liège, Liège, Belgium</nlm:aff><country xml:lang="fr">Belgique</country><wicri:regionArea>Cyclotron Research Centre, University of Liège, Liège</wicri:regionArea><orgName type="university">Université de Liège</orgName><placeName><settlement type="city">Liège</settlement><region type="region" nuts="1">Région wallonne</region><region type="province" nuts="1">Province de Liège</region></placeName></affiliation></author><author><name sortKey="Luxen, Andre" sort="Luxen, Andre" uniqKey="Luxen A" first="André" last="Luxen">André Luxen</name><affiliation wicri:level="4"><nlm:aff id="aff1">Cyclotron Research Centre, University of Liège, Liège, Belgium</nlm:aff><country xml:lang="fr">Belgique</country><wicri:regionArea>Cyclotron Research Centre, University of Liège, Liège</wicri:regionArea><orgName type="university">Université de Liège</orgName><placeName><settlement type="city">Liège</settlement><region type="region" nuts="1">Région wallonne</region><region type="province" nuts="1">Province de Liège</region></placeName></affiliation></author><author><name sortKey="Phillips, Christophe" sort="Phillips, Christophe" uniqKey="Phillips C" first="Christophe" last="Phillips">Christophe Phillips</name><affiliation wicri:level="4"><nlm:aff id="aff1">Cyclotron Research Centre, University of Liège, Liège, Belgium</nlm:aff><country xml:lang="fr">Belgique</country><wicri:regionArea>Cyclotron Research Centre, University of Liège, Liège</wicri:regionArea><orgName type="university">Université de Liège</orgName><placeName><settlement type="city">Liège</settlement><region type="region" nuts="1">Région wallonne</region><region type="province" nuts="1">Province de Liège</region></placeName></affiliation><affiliation wicri:level="4"><nlm:aff id="aff6">Department of Electrical Engineering and Computer Science, University of Liège, Liège, Belgium</nlm:aff><country xml:lang="fr">Belgique</country><wicri:regionArea>Department of Electrical Engineering and Computer Science, University of Liège, Liège</wicri:regionArea><orgName type="university">Université de Liège</orgName><placeName><settlement type="city">Liège</settlement><region type="region" nuts="1">Région wallonne</region><region type="province" nuts="1">Province de Liège</region></placeName></affiliation></author><author><name sortKey="Laureys, Steven" sort="Laureys, Steven" uniqKey="Laureys S" first="Steven" last="Laureys">Steven Laureys</name><affiliation wicri:level="4"><nlm:aff id="aff1">Cyclotron Research Centre, University of Liège, Liège, Belgium</nlm:aff><country xml:lang="fr">Belgique</country><wicri:regionArea>Cyclotron Research Centre, University of Liège, Liège</wicri:regionArea><orgName type="university">Université de Liège</orgName><placeName><settlement type="city">Liège</settlement><region type="region" nuts="1">Région wallonne</region><region type="province" nuts="1">Province de Liège</region></placeName></affiliation><affiliation wicri:level="1"><nlm:aff id="aff2">Coma Science Group, Neurology Department, University Hospital of Liège, Liège, Belgium</nlm:aff><country xml:lang="fr">Belgique</country><wicri:regionArea>Coma Science Group, Neurology Department, University Hospital of Liège, Liège</wicri:regionArea><wicri:noRegion>Liège</wicri:noRegion></affiliation></author></titleStmt><publicationStmt><idno type="wicri:source">PMC</idno><idno type="pmid">24936420</idno><idno type="pmc">4053638</idno><idno type="url">http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4053638</idno><idno type="RBID">PMC:4053638</idno><idno type="doi">10.1016/j.nicl.2014.04.004</idno><date when="2014">2014</date><idno type="wicri:Area/Pmc/Corpus">000A43</idno><idno type="wicri:explorRef" wicri:stream="Pmc" wicri:step="Corpus" wicri:corpus="PMC">000A43</idno><idno type="wicri:Area/Pmc/Curation">000A43</idno><idno type="wicri:explorRef" wicri:stream="Pmc" wicri:step="Curation">000A43</idno><idno type="wicri:Area/Pmc/Checkpoint">000524</idno><idno type="wicri:explorRef" wicri:stream="Pmc" wicri:step="Checkpoint">000524</idno><idno type="wicri:source">PubMed</idno><idno type="wicri:Area/PubMed/Corpus">000768</idno><idno type="wicri:explorRef" wicri:stream="PubMed" wicri:step="Corpus" wicri:corpus="PubMed">000768</idno><idno type="wicri:Area/PubMed/Curation">000768</idno><idno type="wicri:explorRef" wicri:stream="PubMed" wicri:step="Curation">000768</idno><idno type="wicri:Area/PubMed/Checkpoint">000768</idno><idno type="wicri:explorRef" wicri:stream="Checkpoint" wicri:step="PubMed">000768</idno><idno type="wicri:Area/Ncbi/Merge">001796</idno><idno type="wicri:Area/Ncbi/Curation">001796</idno><idno type="wicri:Area/Ncbi/Checkpoint">001796</idno><idno type="wicri:Area/Main/Merge">000B36</idno><idno type="wicri:Area/Main/Curation">000B32</idno><idno type="wicri:Area/Main/Exploration">000B32</idno></publicationStmt><sourceDesc><biblStruct><analytic><title xml:lang="en" level="a" type="main">Biased binomial assessment of cross-validated estimation of classification accuracies illustrated in diagnosis predictions</title><author><name sortKey="Noirhomme, Quentin" sort="Noirhomme, Quentin" uniqKey="Noirhomme Q" first="Quentin" last="Noirhomme">Quentin Noirhomme</name><affiliation wicri:level="4"><nlm:aff id="aff1">Cyclotron Research Centre, University of Liège, Liège, Belgium</nlm:aff><country xml:lang="fr">Belgique</country><wicri:regionArea>Cyclotron Research Centre, University of Liège, Liège</wicri:regionArea><orgName type="university">Université de Liège</orgName><placeName><settlement type="city">Liège</settlement><region type="region" nuts="1">Région wallonne</region><region type="province" nuts="1">Province de Liège</region></placeName></affiliation><affiliation wicri:level="1"><nlm:aff id="aff2">Coma Science Group, Neurology Department, University Hospital of Liège, Liège, Belgium</nlm:aff><country xml:lang="fr">Belgique</country><wicri:regionArea>Coma Science Group, Neurology Department, University Hospital of Liège, Liège</wicri:regionArea><wicri:noRegion>Liège</wicri:noRegion></affiliation></author><author><name sortKey="Lesenfants, Damien" sort="Lesenfants, Damien" uniqKey="Lesenfants D" first="Damien" last="Lesenfants">Damien Lesenfants</name><affiliation wicri:level="4"><nlm:aff id="aff1">Cyclotron Research Centre, University of Liège, Liège, Belgium</nlm:aff><country xml:lang="fr">Belgique</country><wicri:regionArea>Cyclotron Research Centre, University of Liège, Liège</wicri:regionArea><orgName type="university">Université de Liège</orgName><placeName><settlement type="city">Liège</settlement><region type="region" nuts="1">Région wallonne</region><region type="province" nuts="1">Province de Liège</region></placeName></affiliation><affiliation wicri:level="1"><nlm:aff id="aff2">Coma Science Group, Neurology Department, University Hospital of Liège, Liège, Belgium</nlm:aff><country xml:lang="fr">Belgique</country><wicri:regionArea>Coma Science Group, Neurology Department, University Hospital of Liège, Liège</wicri:regionArea><wicri:noRegion>Liège</wicri:noRegion></affiliation></author><author><name sortKey="Gomez, Francisco" sort="Gomez, Francisco" uniqKey="Gomez F" first="Francisco" last="Gomez">Francisco Gomez</name><affiliation wicri:level="1"><nlm:aff id="aff3">Complexus Group, Computer Science Department, Universidad Central de Colombia, Bogotá, Colombia</nlm:aff><country xml:lang="fr">Colombie</country><wicri:regionArea>Complexus Group, Computer Science Department, Universidad Central de Colombia, Bogotá</wicri:regionArea><wicri:noRegion>Bogotá</wicri:noRegion></affiliation></author><author><name sortKey="Soddu, Andrea" sort="Soddu, Andrea" uniqKey="Soddu A" first="Andrea" last="Soddu">Andrea Soddu</name><affiliation wicri:level="1"><nlm:aff id="aff4">Department of Physics & Astronomy, Brain and Mind Institute, University of Western Ontario, London, ON, Canada</nlm:aff><country xml:lang="fr">Canada</country><wicri:regionArea>Department of Physics & Astronomy, Brain and Mind Institute, University of Western Ontario, London, ON</wicri:regionArea><wicri:noRegion>ON</wicri:noRegion></affiliation></author><author><name sortKey="Schrouff, Jessica" sort="Schrouff, Jessica" uniqKey="Schrouff J" first="Jessica" last="Schrouff">Jessica Schrouff</name><affiliation wicri:level="4"><nlm:aff id="aff1">Cyclotron Research Centre, University of Liège, Liège, Belgium</nlm:aff><country xml:lang="fr">Belgique</country><wicri:regionArea>Cyclotron Research Centre, University of Liège, Liège</wicri:regionArea><orgName type="university">Université de Liège</orgName><placeName><settlement type="city">Liège</settlement><region type="region" nuts="1">Région wallonne</region><region type="province" nuts="1">Province de Liège</region></placeName></affiliation><affiliation wicri:level="1"><nlm:aff id="aff5">Laboratory of Behavioral and Cognitive Neurology, Stanford University, Palo Alto, USA</nlm:aff><country xml:lang="fr">États-Unis</country><wicri:regionArea>Laboratory of Behavioral and Cognitive Neurology, Stanford University, Palo Alto</wicri:regionArea><wicri:noRegion>Palo Alto</wicri:noRegion></affiliation></author><author><name sortKey="Garraux, Gaetan" sort="Garraux, Gaetan" uniqKey="Garraux G" first="Gaëtan" last="Garraux">Gaëtan Garraux</name><affiliation wicri:level="4"><nlm:aff id="aff1">Cyclotron Research Centre, University of Liège, Liège, Belgium</nlm:aff><country xml:lang="fr">Belgique</country><wicri:regionArea>Cyclotron Research Centre, University of Liège, Liège</wicri:regionArea><orgName type="university">Université de Liège</orgName><placeName><settlement type="city">Liège</settlement><region type="region" nuts="1">Région wallonne</region><region type="province" nuts="1">Province de Liège</region></placeName></affiliation></author><author><name sortKey="Luxen, Andre" sort="Luxen, Andre" uniqKey="Luxen A" first="André" last="Luxen">André Luxen</name><affiliation wicri:level="4"><nlm:aff id="aff1">Cyclotron Research Centre, University of Liège, Liège, Belgium</nlm:aff><country xml:lang="fr">Belgique</country><wicri:regionArea>Cyclotron Research Centre, University of Liège, Liège</wicri:regionArea><orgName type="university">Université de Liège</orgName><placeName><settlement type="city">Liège</settlement><region type="region" nuts="1">Région wallonne</region><region type="province" nuts="1">Province de Liège</region></placeName></affiliation></author><author><name sortKey="Phillips, Christophe" sort="Phillips, Christophe" uniqKey="Phillips C" first="Christophe" last="Phillips">Christophe Phillips</name><affiliation wicri:level="4"><nlm:aff id="aff1">Cyclotron Research Centre, University of Liège, Liège, Belgium</nlm:aff><country xml:lang="fr">Belgique</country><wicri:regionArea>Cyclotron Research Centre, University of Liège, Liège</wicri:regionArea><orgName type="university">Université de Liège</orgName><placeName><settlement type="city">Liège</settlement><region type="region" nuts="1">Région wallonne</region><region type="province" nuts="1">Province de Liège</region></placeName></affiliation><affiliation wicri:level="4"><nlm:aff id="aff6">Department of Electrical Engineering and Computer Science, University of Liège, Liège, Belgium</nlm:aff><country xml:lang="fr">Belgique</country><wicri:regionArea>Department of Electrical Engineering and Computer Science, University of Liège, Liège</wicri:regionArea><orgName type="university">Université de Liège</orgName><placeName><settlement type="city">Liège</settlement><region type="region" nuts="1">Région wallonne</region><region type="province" nuts="1">Province de Liège</region></placeName></affiliation></author><author><name sortKey="Laureys, Steven" sort="Laureys, Steven" uniqKey="Laureys S" first="Steven" last="Laureys">Steven Laureys</name><affiliation wicri:level="4"><nlm:aff id="aff1">Cyclotron Research Centre, University of Liège, Liège, Belgium</nlm:aff><country xml:lang="fr">Belgique</country><wicri:regionArea>Cyclotron Research Centre, University of Liège, Liège</wicri:regionArea><orgName type="university">Université de Liège</orgName><placeName><settlement type="city">Liège</settlement><region type="region" nuts="1">Région wallonne</region><region type="province" nuts="1">Province de Liège</region></placeName></affiliation><affiliation wicri:level="1"><nlm:aff id="aff2">Coma Science Group, Neurology Department, University Hospital of Liège, Liège, Belgium</nlm:aff><country xml:lang="fr">Belgique</country><wicri:regionArea>Coma Science Group, Neurology Department, University Hospital of Liège, Liège</wicri:regionArea><wicri:noRegion>Liège</wicri:noRegion></affiliation></author></analytic><series><title level="j">NeuroImage : Clinical</title><idno type="eISSN">2213-1582</idno><imprint><date when="2014">2014</date></imprint></series></biblStruct></sourceDesc></fileDesc><profileDesc><textClass><keywords scheme="KwdEn" xml:lang="en"><term>Adult</term><term>Aged</term><term>Bias (Epidemiology)</term><term>Brain (pathology)</term><term>Brain Injuries (diagnosis)</term><term>Brain-Computer Interfaces</term><term>Computer Simulation</term><term>Humans</term><term>Middle Aged</term><term>Models, Statistical</term><term>Young Adult</term></keywords><keywords scheme="MESH" qualifier="diagnosis" xml:lang="en"><term>Brain Injuries</term></keywords><keywords scheme="MESH" qualifier="pathology" xml:lang="en"><term>Brain</term></keywords><keywords scheme="MESH" xml:lang="en"><term>Adult</term><term>Aged</term><term>Bias (Epidemiology)</term><term>Brain-Computer Interfaces</term><term>Computer Simulation</term><term>Humans</term><term>Middle Aged</term><term>Models, Statistical</term><term>Young Adult</term></keywords></textClass></profileDesc></teiHeader><front><div type="abstract" xml:lang="en"><p>Multivariate classification is used in neuroimaging studies to infer brain activation or in medical applications to infer diagnosis. Their results are often assessed through either a binomial or a permutation test. Here, we simulated classification results of generated random data to assess the influence of the cross-validation scheme on the significance of results. Distributions built from classification of random data with cross-validation did not follow the binomial distribution. The binomial test is therefore not adapted. On the contrary, the permutation test was unaffected by the cross-validation scheme. The influence of the cross-validation was further illustrated on real-data from a brain–computer interface experiment in patients with disorders of consciousness and from an fMRI study on patients with Parkinson disease. Three out of 16 patients with disorders of consciousness had significant accuracy on binomial testing, but only one showed significant accuracy using permutation testing. In the fMRI experiment, the mental imagery of gait could discriminate significantly between idiopathic Parkinson's disease patients and healthy subjects according to the permutation test but not according to the binomial test. Hence, binomial testing could lead to biased estimation of significance and false positive or negative results. In our view, permutation testing is thus recommended for clinical application of classification with cross-validation.</p></div></front><back><div1 type="bibliography"><listBibl><biblStruct><analytic><author><name sortKey="Benjamini, Y" uniqKey="Benjamini Y">Y. Benjamini</name></author><author><name sortKey="Hochberg, Y" uniqKey="Hochberg Y">Y. Hochberg</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Berrar, D" uniqKey="Berrar D">D. Berrar</name></author><author><name sortKey="Bradbury, I" uniqKey="Bradbury I">I. Bradbury</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Billinger, M" uniqKey="Billinger M">M. Billinger</name></author><author><name sortKey="Daly, I" uniqKey="Daly I">I. Daly</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Burges, C" uniqKey="Burges C">C. Burges</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Chang, C C" uniqKey="Chang C">C.-C. Chang</name></author><author><name sortKey="Lin, C J" uniqKey="Lin C">C.-J. Lin</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Cremers, J" uniqKey="Cremers J">J. Cremers</name></author><author><name sortKey="D Ostilio, K" uniqKey="D Ostilio K">K. D'Ostilio</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Cruse, D" uniqKey="Cruse D">D. Cruse</name></author><author><name sortKey="Chennu, S" uniqKey="Chennu S">S. Chennu</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Donchin, E" uniqKey="Donchin E">E. Donchin</name></author><author><name sortKey="Spencer, K M" uniqKey="Spencer K">K.M. Spencer</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Efron, B" uniqKey="Efron B">B. Efron</name></author><author><name sortKey="Tibshirani, R" uniqKey="Tibshirani R">R. Tibshirani</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Etzel, J A" uniqKey="Etzel J">J.A. Etzel</name></author><author><name sortKey="Gazzola, V" uniqKey="Gazzola V">V. Gazzola</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Farwell, L A" uniqKey="Farwell L">L.A. Farwell</name></author><author><name sortKey="Donchin, E" uniqKey="Donchin E">E. Donchin</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Focke, N K" uniqKey="Focke N">N.K. Focke</name></author><author><name sortKey="Helms, G" uniqKey="Helms G">G. Helms</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Furdea, A" uniqKey="Furdea A">A. Furdea</name></author><author><name sortKey="Halder, S" uniqKey="Halder S">S. Halder</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Galanaud, D" uniqKey="Galanaud D">D. Galanaud</name></author><author><name sortKey="Perlbarg, V" uniqKey="Perlbarg V">V. Perlbarg</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Garraux, G" uniqKey="Garraux G">G. Garraux</name></author><author><name sortKey="Phillips, C" uniqKey="Phillips C">C. Phillips</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Goldfine, A M" uniqKey="Goldfine A">A.M. Goldfine</name></author><author><name sortKey="Bardin, J C" uniqKey="Bardin J">J.C. Bardin</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Golub, T R" uniqKey="Golub T">T.R. Golub</name></author><author><name sortKey="Slonim, D K" uniqKey="Slonim D">D.K. Slonim</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Good, P" uniqKey="Good P">P. Good</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Howell, D C" uniqKey="Howell D">D.C. Howell</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Hughes, A J" uniqKey="Hughes A">A.J. Hughes</name></author><author><name sortKey="Daniel, S E" uniqKey="Daniel S">S.E. Daniel</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Kohavi, R" uniqKey="Kohavi R">R. Kohavi</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Krusienski, D J" uniqKey="Krusienski D">D.J. Krusienski</name></author><author><name sortKey="Sellers, E W" uniqKey="Sellers E">E.W. Sellers</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Kubler, A" uniqKey="Kubler A">A. Kubler</name></author><author><name sortKey="Birbaumer, N" uniqKey="Birbaumer N">N. Birbaumer</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Laureys, S" uniqKey="Laureys S">S. Laureys</name></author><author><name sortKey="Schiff, N D" uniqKey="Schiff N">N.D. Schiff</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Lemm, S" uniqKey="Lemm S">S. Lemm</name></author><author><name sortKey="Blankertz, B" uniqKey="Blankertz B">B. Blankertz</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Lule, D" uniqKey="Lule D">D. Lule</name></author><author><name sortKey="Noirhomme, Q" uniqKey="Noirhomme Q">Q. Noirhomme</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Luyt, C E" uniqKey="Luyt C">C.E. Luyt</name></author><author><name sortKey="Galanaud, D" uniqKey="Galanaud D">D. Galanaud</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Maillet, A" uniqKey="Maillet A">A. Maillet</name></author><author><name sortKey="Pollak, P" uniqKey="Pollak P">P. Pollak</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Maris, E" uniqKey="Maris E">E. Maris</name></author><author><name sortKey="Oostenveld, R" uniqKey="Oostenveld R">R. Oostenveld</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Martin, J K" uniqKey="Martin J">J.K. Martin</name></author><author><name sortKey="Hirschberg, D S" uniqKey="Hirschberg D">D.S. Hirschberg</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Mukherjee, S" uniqKey="Mukherjee S">S. Mukherjee</name></author><author><name sortKey="Golland, P" uniqKey="Golland P">P. Golland</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Muller Putz, G R" uniqKey="Muller Putz G">G.R. Müller-Putz</name></author><author><name sortKey="Scherer, R" uniqKey="Scherer R">R. Scherer</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Nichols, T E" uniqKey="Nichols T">T.E. Nichols</name></author><author><name sortKey="Holmes, A P" uniqKey="Holmes A">A.P. Holmes</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Ojala, M" uniqKey="Ojala M">M. Ojala</name></author><author><name sortKey="Garriga, G C" uniqKey="Garriga G">G.C. Garriga</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Orru, G" uniqKey="Orru G">G. Orru</name></author><author><name sortKey="Pettersson Yeo, W" uniqKey="Pettersson Yeo W">W. Pettersson-Yeo</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Pereira, F" uniqKey="Pereira F">F. Pereira</name></author><author><name sortKey="Botvinick, M" uniqKey="Botvinick M">M. Botvinick</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Pereira, F" uniqKey="Pereira F">F. Pereira</name></author><author><name sortKey="Detre, G" uniqKey="Detre G">G. Detre</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Pereira, F" uniqKey="Pereira F">F. Pereira</name></author><author><name sortKey="Mitchell, T" uniqKey="Mitchell T">T. Mitchell</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Phillips, C L" uniqKey="Phillips C">C.L. Phillips</name></author><author><name sortKey="Bruno, M A" uniqKey="Bruno M">M.A. Bruno</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Picard, R R" uniqKey="Picard R">R.R. Picard</name></author><author><name sortKey="Cook, R D" uniqKey="Cook R">R.D. Cook</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Schalk, G" uniqKey="Schalk G">G. Schalk</name></author><author><name sortKey="Mcfarland, D J" uniqKey="Mcfarland D">D.J. McFarland</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Schrouff, J" uniqKey="Schrouff J">J. Schrouff</name></author><author><name sortKey="Cremers, J" uniqKey="Cremers J">J. Cremers</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Schrouff, J" uniqKey="Schrouff J">J. Schrouff</name></author><author><name sortKey="Cremers, J" uniqKey="Cremers J">J. Cremers</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Schrouff, J" uniqKey="Schrouff J">J. Schrouff</name></author><author><name sortKey="Rosa, M J" uniqKey="Rosa M">M.J. Rosa</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Sellers, E W" uniqKey="Sellers E">E.W. Sellers</name></author><author><name sortKey="Donchin, E" uniqKey="Donchin E">E. Donchin</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Sharbrough, F" uniqKey="Sharbrough F">F. Sharbrough</name></author><author><name sortKey="Chatrian, G E" uniqKey="Chatrian G">G.-E. Chatrian</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Simon, R" uniqKey="Simon R">R. Simon</name></author><author><name sortKey="Radmacher, M D" uniqKey="Radmacher M">M.D. Radmacher</name></author></analytic></biblStruct><biblStruct></biblStruct></listBibl></div1></back></TEI><affiliations><list><country><li>Belgique</li><li>Canada</li><li>Colombie</li><li>États-Unis</li></country><region><li>Province de Liège</li><li>Région wallonne</li></region><settlement><li>Liège</li></settlement><orgName><li>Université de Liège</li></orgName></list><tree><country name="Belgique"><region name="Région wallonne"><name sortKey="Noirhomme, Quentin" sort="Noirhomme, Quentin" uniqKey="Noirhomme Q" first="Quentin" last="Noirhomme">Quentin Noirhomme</name></region><name sortKey="Garraux, Gaetan" sort="Garraux, Gaetan" uniqKey="Garraux G" first="Gaëtan" last="Garraux">Gaëtan Garraux</name><name sortKey="Laureys, Steven" sort="Laureys, Steven" uniqKey="Laureys S" first="Steven" last="Laureys">Steven Laureys</name><name sortKey="Laureys, Steven" sort="Laureys, Steven" uniqKey="Laureys S" first="Steven" last="Laureys">Steven Laureys</name><name sortKey="Lesenfants, Damien" sort="Lesenfants, Damien" uniqKey="Lesenfants D" first="Damien" last="Lesenfants">Damien Lesenfants</name><name sortKey="Lesenfants, Damien" sort="Lesenfants, Damien" uniqKey="Lesenfants D" first="Damien" last="Lesenfants">Damien Lesenfants</name><name sortKey="Luxen, Andre" sort="Luxen, Andre" uniqKey="Luxen A" first="André" last="Luxen">André Luxen</name><name sortKey="Noirhomme, Quentin" sort="Noirhomme, Quentin" uniqKey="Noirhomme Q" first="Quentin" last="Noirhomme">Quentin Noirhomme</name><name sortKey="Phillips, Christophe" sort="Phillips, Christophe" uniqKey="Phillips C" first="Christophe" last="Phillips">Christophe Phillips</name><name sortKey="Phillips, Christophe" sort="Phillips, Christophe" uniqKey="Phillips C" first="Christophe" last="Phillips">Christophe Phillips</name><name sortKey="Schrouff, Jessica" sort="Schrouff, Jessica" uniqKey="Schrouff J" first="Jessica" last="Schrouff">Jessica Schrouff</name></country><country name="Colombie"><noRegion><name sortKey="Gomez, Francisco" sort="Gomez, Francisco" uniqKey="Gomez F" first="Francisco" last="Gomez">Francisco Gomez</name></noRegion></country><country name="Canada"><noRegion><name sortKey="Soddu, Andrea" sort="Soddu, Andrea" uniqKey="Soddu A" first="Andrea" last="Soddu">Andrea Soddu</name></noRegion></country><country name="États-Unis"><noRegion><name sortKey="Schrouff, Jessica" sort="Schrouff, Jessica" uniqKey="Schrouff J" first="Jessica" last="Schrouff">Jessica Schrouff</name></noRegion></country></tree></affiliations></record>Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Wicri/Canada/explor/ParkinsonCanadaV1/Data/Main/Exploration
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 000B32 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Main/Exploration/biblio.hfd -nk 000B32 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien
|wiki= Wicri/Canada
|area= ParkinsonCanadaV1
|flux= Main
|étape= Exploration
|type= RBID
|clé= PMC:4053638
|texte= Biased binomial assessment of cross-validated estimation of classification accuracies illustrated in diagnosis predictions
}}
Pour générer des pages wiki
HfdIndexSelect -h $EXPLOR_AREA/Data/Main/Exploration/RBID.i -Sk "pubmed:24936420" \
| HfdSelect -Kh $EXPLOR_AREA/Data/Main/Exploration/biblio.hfd \
| NlmPubMed2Wicri -a ParkinsonCanadaV1
| This area was generated with Dilib version V0.6.29. |  |